$$BasicStatistics$$$$BasicStatistics$$BasicStatisticsBasic\,Statistics$$BasicStatistics$$

Measures of Central Tendency

Moments

• •$\mathbf{\text{First Moment (Mean):}}$$\mathbf{\text{First Moment (Mean):}}$"First Moment (Mean):"\textbf{First Moment (Mean):}$\mathbf{\text{First Moment (Mean):}}$ The first moment is the arithmetic mean of a dataset and is denoted by $\mu$$\mu$mu\mu$\mu$ . It represents the center of mass or average value of the data points.
  
  Mathematically, the mean $\mu$$\mu$mu\mu$\mu$ of a dataset with $n$$n$nn$n$ data points $\{x_1,x_2,...,x_n\}$$\{x_1,x_2,...,x_n\}${x_(1),x_(2),...,x_(n)}\{x_1, x_2, ..., x_n\}$\{x_1,x_2,...,x_n\}$ is calculated as:
  
  $$\mu =\frac{1}{n}\sum _{i=1}^n{x}_i$$$$\mu =\frac{1}{n}\sum _{i=1}^n x_i$$mu=(1)/(n)sum_(i=1)^(n)x_(i)\begin{equation*} \mu = \frac{1}{n} \sum_{i=1}^{n}x_i \end{equation*}$$\mu =\frac{1}{n}\sum _{i=1}^n{x}_i$$
  For a continuous random variable $X$$X$XX$X$ with probability density function $f(x)$$f(x)$f(x)f(x)$f(x)$, the first moment about the origin (mean) is given by:
  
  $$\mu =\int x\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$$$\mu =\int x\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$mu=intx*f(x)dxquad"from"-oo"to"+oo\mu = \int{x \cdot f(x) \, dx} \quad \text{from} \, -\infty \, \text{to} \, +\infty$$\mu =\int x\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$
• •$\mathbf{\text{Second Moment (Variance):}}$$\mathbf{\text{Second Moment (Variance):}}$"Second Moment (Variance):"\textbf{Second Moment (Variance):}$\mathbf{\text{Second Moment (Variance):}}$ The second moment measures the spread or dispersion of the data points around the mean. It is denoted by ${\sigma }^2$${\sigma }^2$sigma^(2)\sigma^2${\sigma }^2$ (sigma squared) and is the average of the squared differences between each data point and the mean.
  
  Mathematically, the variance ${\sigma }^2$${\sigma }^2$sigma^(2)\sigma^2${\sigma }^2$ of a dataset with $n$$n$nn$n$ data points $\{x_1,x_2,...,x_n\}$$\{x_1,x_2,...,x_n\}${x_(1),x_(2),...,x_(n)}\{x_1, x_2, ..., x_n\}$\{x_1,x_2,...,x_n\}$ and mean $\mu$$\mu$mu\mu$\mu$ is calculated as:
  
  $${\sigma }^2=\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2$$$${\sigma }^2=\frac{1}{n}\sum _{i=1}^n (x_i-\mu {)}^2$$sigma^(2)=(1)/(n)sum_(i=1)^(n)(x_(i)-mu)^(2)\begin{equation*} \sigma^2 = \frac{1}{n} \sum_{i=1}^{n}(x_i-\mu)^2 \end{equation*}$${\sigma }^2=\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2$$
  For a continuous random variable $X$$X$XX$X$ with probability density function $f(x)$$f(x)$f(x)f(x)$f(x)$, the second moment about the origin (variance) is given by:
  
  $${\sigma }^2=\int (x-\mu {)}^2\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$$${\sigma }^2=\int (x-\mu {)}^2\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$sigma^(2)=int(x-mu)^(2)*f(x)dxquad"from"-oo"to"+oo\sigma^2 = \int{(x - \mu)^2 \cdot f(x) \, dx} \quad \text{from} \, -\infty \, \text{to} \, +\infty$${\sigma }^2=\int (x-\mu {)}^2\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$
• •$\mathbf{\text{Third Moment (Skewness):}}$$\mathbf{\text{Third Moment (Skewness):}}$"Third Moment (Skewness):"\textbf{Third Moment (Skewness):}$\mathbf{\text{Third Moment (Skewness):}}$ Skewness measures the asymmetry of the data distribution. It indicates whether the data is skewed to the left (negative skewness) or to the right (positive skewness) compared to the mean. Skewness is denoted by ${\gamma }_1$${\gamma }_1$gamma_(1)\gamma_1${\gamma }_1$ (gamma one).
  
  Mathematically, the skewness ${\gamma }_1$${\gamma }_1$gamma_(1)\gamma_1${\gamma }_1$ of a dataset with $n$$n$nn$n$ data points $\{x_1,x_2,...,x_n\}$$\{x_1,x_2,...,x_n\}${x_(1),x_(2),...,x_(n)}\{x_1, x_2, ..., x_n\}$\{x_1,x_2,...,x_n\}$ and mean $\mu$$\mu$mu\mu$\mu$ is calculated as:
  
  $${\gamma }_1=\frac{\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^3}{{(\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2)}^{\frac{3}{2}}}$$$${\gamma }_1=\frac{\frac{1}{n}\sum _{i=1}^n (x_i-\mu {)}^3}{{\left(\frac{1}{n}\sum _{i=1}^n (x_i-\mu {)}^2\right)}^{\frac{3}{2}}}$$gamma_(1)=((1)/(n)sum_(i=1)^(n)(x_(i)-mu)^(3))/(((1)/(n)sum_(i=1)^(n)(x_(i)-mu)^(2))^((3)/(2)))\begin{equation*} \gamma_1 = \frac{\frac{1}{n}\sum_{i=1}^{n}(x_i-\mu)^3}{\left(\frac{1}{n}\sum_{i=1}^{n}(x_i-\mu)^2\right)^{\frac{3}{2}}} \end{equation*}$${\gamma }_1=\frac{\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^3}{{(\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2)}^{\frac{3}{2}}}$$
  For a continuous random variable $X$$X$XX$X$ with probability density function $f(x)$$f(x)$f(x)f(x)$f(x)$, the third central moment (related to skewness) is given by:
  
  $${\gamma }_3=\int (x-\mu {)}^3\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$$${\gamma }_3=\int (x-\mu {)}^3\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$gamma_(3)=int(x-mu)^(3)*f(x)dxquad"from"-oo"to"+oo\gamma_3 = \int{(x - \mu)^3 \cdot f(x) \, dx} \quad \text{from} \, -\infty \, \text{to} \, +\infty$${\gamma }_3=\int (x-\mu {)}^3\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$
• •$\mathbf{\text{Fourth Moment (Kurtosis):}}$$\mathbf{\text{Fourth Moment (Kurtosis):}}$"Fourth Moment (Kurtosis):"\textbf{Fourth Moment (Kurtosis):}$\mathbf{\text{Fourth Moment (Kurtosis):}}$ Kurtosis measures the shape of the distribution and assesses the presence of outliers or heavy tails. High kurtosis indicates a sharper peak and heavier tails, while low kurtosis indicates a flatter peak and lighter tails. Kurtosis is denoted by ${\gamma }_2$${\gamma }_2$gamma_(2)\gamma_2${\gamma }_2$ (gamma two).
  
  Mathematically, the kurtosis ${\gamma }_2$${\gamma }_2$gamma_(2)\gamma_2${\gamma }_2$ of a dataset with $n$$n$nn$n$ data points $\{x_1,x_2,...,x_n\}$$\{x_1,x_2,...,x_n\}${x_(1),x_(2),...,x_(n)}\{x_1, x_2, ..., x_n\}$\{x_1,x_2,...,x_n\}$ and mean $\mu$$\mu$mu\mu$\mu$ is calculated as:
  
  $${\gamma }_2=\frac{\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^4}{{(\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2)}^2}$$$${\gamma }_2=\frac{\frac{1}{n}\sum _{i=1}^n (x_i-\mu {)}^4}{{\left(\frac{1}{n}\sum _{i=1}^n (x_i-\mu {)}^2\right)}^2}$$gamma_(2)=((1)/(n)sum_(i=1)^(n)(x_(i)-mu)^(4))/(((1)/(n)sum_(i=1)^(n)(x_(i)-mu)^(2))^(2))\begin{equation*} \gamma_2 = \frac{\frac{1}{n}\sum_{i=1}^{n}(x_i-\mu)^4}{\left(\frac{1}{n}\sum_{i=1}^{n}(x_i-\mu)^2\right)^2} \end{equation*}$${\gamma }_2=\frac{\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^4}{{(\frac{1}{n}\sum _{i=1}^n(x_i-\mu {)}^2)}^2}$$
  For a continuous random variable $X$$X$XX$X$ with probability density function $f(x)$$f(x)$f(x)f(x)$f(x)$, the fourth central moment (related to kurtosis) is given by:
  
  $${\gamma }_4=\int (x-\mu {)}^4\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$$${\gamma }_4=\int (x-\mu {)}^4\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$gamma_(4)=int(x-mu)^(4)*f(x)dxquad"from"-oo"to"+oo\gamma_4 = \int{(x - \mu)^4 \cdot f(x) \, dx} \quad \text{from} \, -\infty \, \text{to} \, +\infty$${\gamma }_4=\int (x-\mu {)}^4\cdot f(x)dx\text{from}-\mathrm{\infty }\text{to}+\mathrm{\infty }$$

Skewness

• •$\mathbf{\text{Positive Skewness:}}$$\mathbf{\text{Positive Skewness:}}$"Positive Skewness:"\textbf{Positive Skewness:}$\mathbf{\text{Positive Skewness:}}$ A positive skewness value indicates that the data is skewed to the right, meaning the tail on the right side of the distribution is longer, and the majority of the data points are concentrated on the left side.
• •$\mathbf{\text{Negative Skewness:}}$$\mathbf{\text{Negative Skewness:}}$"Negative Skewness:"\textbf{Negative Skewness:}$\mathbf{\text{Negative Skewness:}}$ A negative skewness value indicates that the data is skewed to the left, meaning the tail on the left side of the distribution is longer, and the majority of the data points are concentrated on the right side.
• •$\mathbf{\text{Zero Skewness:}}$$\mathbf{\text{Zero Skewness:}}$"Zero Skewness:"\textbf{Zero Skewness:}$\mathbf{\text{Zero Skewness:}}$ A skewness value close to zero suggests that the data is approximately symmetrically distributed.

Kurtosis

• •$\mathbf{\text{Leptokurtic:}}$$\mathbf{\text{Leptokurtic:}}$"Leptokurtic:"\textbf{Leptokurtic:}$\mathbf{\text{Leptokurtic:}}$ Positive excess kurtosis (${\gamma }_2>0$${\gamma }_2>0$gamma_(2) > 0\gamma_2 > 0${\gamma }_2>0$) indicates a leptokurtic distribution, which has heavier tails and a sharper peak compared to a normal distribution.
• •$\mathbf{\text{Mesokurtic:}}$$\mathbf{\text{Mesokurtic:}}$"Mesokurtic:"\textbf{Mesokurtic:}$\mathbf{\text{Mesokurtic:}}$ A normal distribution has zero excess kurtosis (${\gamma }_2=0$${\gamma }_2=0$gamma_(2)=0\gamma_2 = 0${\gamma }_2=0$) and is referred to as mesokurtic. It means the tails are similar to those of a normal distribution.
• •$\mathbf{\text{Platykurtic:}}$$\mathbf{\text{Platykurtic:}}$"Platykurtic:"\textbf{Platykurtic:}$\mathbf{\text{Platykurtic:}}$ Negative excess kurtosis (${\gamma }_2<0$${\gamma }_2<0$gamma_(2) < 0\gamma_2 < 0${\gamma }_2<0$) indicates a platykurtic distribution, which has lighter tails and a flatter peak compared to a normal distribution.

Correlation

Pearson Correlation Coefficient

• •$r=1$$r=1$r=1r = 1$r=1$ indicates a perfect positive correlation (both variables increase together linearly).
• •$r=-1$$r=-1$r=-1r = -1$r=-1$ indicates a perfect negative correlation (as one variable increases, the other decreases linearly).
• •$r=0$$r=0$r=0r = 0$r=0$ indicates no linear correlation (the variables are not linearly related).

Spearman and Kendall Correlation

Interpreting Correlation

Regression

Linear Regression

• •$y$$y$yy$y$ is the dependent variable,
• •$x$$x$xx$x$ is the independent variable,
• •${\beta }_0$${\beta }_0$beta_(0)\beta_0${\beta }_0$ is the intercept (y-value when $x$$x$xx$x$ is 0),
• •${\beta }_1$${\beta }_1$beta_(1)\beta_1${\beta }_1$ is the slope (change in $y$$y$yy$y$ for a unit change in $x$$x$xx$x$),
• •$\epsilon$$\epsilon$epsi\varepsilon$\epsilon$ represents the error term (residuals, the difference between the predicted and actual values).

Multiple Regression

Interpreting Regression

Assumptions

Important Note

Rank Correlation

Spearman Rank Correlation Coefficient (Spearman's $\rho$$\rho$rho\rho$\rho$)

• •$\rho =1$$\rho =1$rho=1\rho = 1$\rho =1$ indicates a perfect positive rank correlation (the ranks of both variables are exactly the same).
• •$\rho =-1$$\rho =-1$rho=-1\rho = -1$\rho =-1$ indicates a perfect negative rank correlation (the ranks of one variable are the reverse of the ranks of the other variable).
• •$\rho =0$$\rho =0$rho=0\rho = 0$\rho =0$ indicates no rank correlation (the ranks are independent of each other).

Kendall Rank Correlation Coefficient (Kendall's $\tau$$\tau$tau\tau$\tau$)

• •$\tau =1$$\tau =1$tau=1\tau = 1$\tau =1$ indicates a perfect positive rank correlation (all data pairs are concordant).
• •$\tau =-1$$\tau =-1$tau=-1\tau = -1$\tau =-1$ indicates a perfect negative rank correlation (all data pairs are discordant).
• •$\tau =0$$\tau =0$tau=0\tau = 0$\tau =0$ indicates no rank correlation (the number of concordant and discordant pairs are equal).